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Abstract 

Random coding arguments are the backbone of most channel capacity achievability proofs. In this paper, we show 
that in their standard form, such arguments are insufficient for proving some network capacity theorems: structured 
coding arguments, such as random linear or lattice codes, attain higher rates. Historically, structured codes have been 
studied as a stepping stone to practical constructions. However, Korner and Marton demonstrated their usefulness 
for capacity theorems through the derivation of the optimal rate region of a distributed functional source coding 
problem. Here, we use multicasting over finite field and Gaussian multiple-access networks as canonical examples 
to demonstrate that even if we want to send bits over a network, structured codes succeed where simple random 
D ' codes fail. Beyond network coding, we also consider distributed computation over noisy channels and a special 

relay-type problem. 

I. Introduction 

Random coding arguments are at the foundation of most channel capacity achievability proofs. The basic idea 
^ ■ (for a multiterminal problem) is as follows. Choose several random variables with an appropriate joint distribution, 
i—i \ Then, generate high-dimensional codebooks with entries drawn i.i.d. according to this joint distribution. Finally, 
analyze the error performance of the codebooks in expectation and use this to show the existence of at least one 
j> ■ good fixed set of codebooks. This method takes us quite far in network information theory. It has been successfully 
CN [ used to establish the capacity region of the multiple-access channel [1], [2], stochastically degraded broadcast 
channel [3], and physically degraded relay channel [4], just to name a few. However, an elegant multiterminal 
q ■ problem developed by Korner and Marton showed that purely random code constructions are not always sufficient 
[5]; structured random codes, such as linear or lattice codes, may be required on the achievability side of the proof. 
O This key insight is the inspiration for this paper. 

Structured random codes are usually considered to shed light on issues related to practical constructions. Given 
a capacity theorem, it is often of interest to demonstrate the existence of a capacity-achieving linear code to show 
that the codebook size (or complexity) need not be exponential in the blocklength. Linearity also often enables 
^ ■ many complexity-saving reductions in decoding algorithms. However, the conditions for structured random codes 
to be capacity-achieving are often more restrictive than those for unrestricted random codes. For instance, linear 
codes achieve capacity for point-to-point channels only when the noise is symmetric [6], [7]. Thus, it is tempting 
to believe that random codes are a strictly more powerful tool for proving capacity theorems. We will show that, 
in a network setting, structured random codes can be more powerful than purely random codes. 



OO 

O 



A. Prior Work 

To the best of our knowledge, the first example of a scenario where structured codes outperform random codes 
was given by Korner and Marton in [5]. In their consideration, a central decoder wants to reconstruct the parity of 
two correlated sources seen by separate encoders. By using linear codes at the encoders, they were able to access 
the full rate region. Note that gains are only seen in this problem over random codes when the underlying sources 
are correlated. In [8], we showed that there are large gains, proportional to the number of users, for computing 
functions over multiple-access channels with structured codes, regardless of the source dependencies. We applied 
these results to joint source-channel sensor network scenarios in [9], [10]. 
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Recently, Krithivasan and Pradhan showed that lattice coding results in an improved inner bound for the distributed 
compression of a linear function of jointly Gaussian sources [11]. In particular, for two positively correlated Gaussian 
sources, recovering the difference requires a lower sum rate if one uses a lattice code. 

For the above scenarios, the computation of a function is directly called for in the problem statement, so structured 
codes seem to be a natural fit. Somewhat surprisingly, if we are only required to send bits across a network, 
structured codes can still provide gains. Some instances of this phenomenon, which we term structural gain, 
have been independently studied by several groups. One interesting direction is using lattices for interference 
cancellation. Philosof, Khisti, Erez, and Zamir demonstrated that for a "dirty" multiple-access channel with two 
additive interferences, one known non-causally at each encoder, dirty paper coding can be successfully implemented 
with a lattice code but not with an purely random code [12]. Recently, Cadambe and Jafar [13] proposed the concept 
of interference alignment. In essence, encoders use their knowledge of the channel to align their transmissions so 
that for each receiver all interferences lie in one subspace and the desired signal lies on another. For a many- 
to-one interference channel, Bresler, Parekh and Tse use a lattice to ensure that all the interferences seen at the 
relevant decoder are aligned [14]. It was also suggested in [15] that structured codes will be useful for implementing 
interference alignment and similar schemes in general multi-user networks. Finally, Sanderovich, Peleg, and Shamai 
find a scaling law for distributed interference cancellation using lattices in [16]. 

Another instance of structural gain is seen in network coding for wireless networks. Instead of avoiding inter- 
ference from other users, we can use a structured code to compute functions reliably on multiple-access channels. 
We developed such network coding strategies for discrete channels in [17] and extended our ideas to the Gaussian 
case in [18]. Narayanan, Wilson, and Sprintson use similar techniques for the two-way relay channel to allow the 
relay to decode and retransmit only the sum of the transmitted messages [19]. This strategy was extended to the 
two-way relay channel with unequal transmit powers by Nam, Chung, and Lee [20]. 

Structured codes are also useful for parallel relay networks with non-white noise spectra. It has been shown that 
amplify-and-forward is asymptotically optimal for a parallel relay network with white noise and that it significantly 
outperforms standard random coding strategies [21]. Using a lattice scheme, Kochman, Khina, Erez, and Zamir 
have managed to attain similar gains for arbitrary noise spectra [22]. 

B. Paper Organization 

In this paper, we survey several of our recent capacity results based on structured codes to show that even if 
we simply want to send bits across a network, random coding arguments are not enoughQ Our main problem 
formulation is inspired by the recent literature on network coding. Essentially, a single sender must multicast 
messages to several receivers over a graph of point-to-point and multiple-access channels. We use our recent work 
on computing functions over multiple-access channels as part of an overall network code. These computation codes 
are inherently structured and efficiently convert noisy multiple-access channels into reliable computational units 
[8]. We will also consider a relay-type problem with multiple senders and a single receiver. 

The paper is organized as follows. In Section [TTJ we give definitions, particularly for structured codes. We review 
the best known results for point-to-point channel coding with structured codes in Section |III] We describe results on 
distributed computation using structured codes in Section|lVl including the Korner-Marton problem and computation 
over multiple-access channels. In Section [Vj, we examine a variant of the relay problem. In Section [V]J we develop 
our network coding problem for additive noise multiple-access networks over finite fields as well as the Gaussian 
case. Finally, in Section IVIII we discuss open problems and future directions. 

II. Definitions 

We now describe the notions of a purely random code and a structured random code. In later sections, we will 
show examples of problems for which structured random codes perform well in expectation while purely random 
codes do not. This enables us to show the existence of at least one good codebook since at least some non-vanishing 
fraction of the randomly generated codebooks must be good in order to keep the expectation good. 

Definition 1: Choose an alphabet X. An (n,R) code, C, is a set of 2 nR distinct length-n vectors in X n . Each 
vector, c 6 C, is referred to as a codeword. 

'A preliminary version of this paper appeared in the 2007 Information Theory Workshop, Lake Tahoe, CA [23]. 
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We now give an informal definition of what we mean by a purely random coding argument. Unfortunately, it is 
quite difficult to give a formal definition that includes all current techniques (such as superposition coding) but 
excludes roundabout ways of constructing a set of linear codes by complicated thinning arguments. 

A purely random code is a code whose codewords are generated according to a distribution such that the elements 
of each codeword are independent and identically distributed (i.i.d.). For example, for an (n, R) purely random 
code for a point-to-point channel, we choose a probability distribution function (pdf) p(x) and draw each of the 
2 nR length-n codewords independently according to p{x n ) = \Yi = iP(x). 

The performance of a random code is often evaluated in expectation under a decoding rule such as maximum- 
likelihood or joint typicality. Given that it performs well in expectation over the random codebook it is then argued 
that at least one good fixed codebook must exist. 

In a multi-user communication problem, more than one codebook is required. Here, it is often useful to allow 
for some dependence between codebooks but for a purely random code one would still generate them elementwise 
i.i.d. according to a joint pdf. For instance, for two users whose codebooks each have 2 nR elements, we can choose 
a distribution p(x±, x 2 ). Then, we draw pairs of codewords independently according to p(x™, = YYi=i Pi x i> x 2)- 

The basic multi-user random coding construction above has been extended to several powerful generalizations in 
the information theory literature. These include random binning [24], block Markov coding [4], [25], superposition 
coding [3], [26], and compress-and-forward [4]. With these tools in hand, most of the currently known achievability 
results of network information theory can be derived. We collectively refer to these strategies as purely random 
coding as they focus on creating codes whose elements have the desired dependencies but pay no attention to their 
algebraic structure. 

Given the large number of such strategies, it is extremely hard to bound the performance of all possible 
purely random coding arguments. For the scope of this paper, we limit ourselves to comparisons to the expected 
performance of the best known purely random coding argument for a particular problem class. For example, for a 
distributed lossless compression problem (such as the Slepian-Wolf problem [27]), we compare to the performance 
of random binning in expectation. 

We now turn to codes that allow for the design of both the codebook pdf and algebraic structure. A structured 
random code (or structured code) is a code that is randomly generated in a way that ensures a particular algebraic 
structure. Given two codewords in a structured code, there is a fixed function that can be applied symbolwise such 
that the resulting vector is also a codeword. For instance, to ensure linearity, we can choose symbolwise addition as 
the fixed function. As with a purely random code, the performance of a structured code is evaluated in expectation 
over the codebooks. In the next two subsections, we provide details on two well-studied classes of structured codes: 
linear codes and lattice codes. 



A. Linear Codes 

Linear codes are the most commonly used structured codes as they often enable many complexity reductions in 
the encoding and decoding algorithms. 

Definition 2: Let F be a finite field. An (n, R) linear code, C, is a structured code on F n that is closed under the 
additive operation of F. That is, given any two codewords, Ci,C2 G C, adding the codewords symbolwise results 
in another codeword: 

Ci = (xn,xi2, • • • ,x ln ) G C (1) 

C2 = (X 2 1,X 2 2, • • • ,X 2n ) G C (2) 
C* = (Xn + X 2 1, X 12 + X 2 2, ...,X ln + X 2n ) (3) 

=► c* G C (4) 

A random linear code is just a linear code drawn according to some distribution. For every linear code there is at 
least one matrix H G ¥ nRxn such that each codeword, c G C C F n , can be written as xH for some x G ¥ nR . 



B. Lattice Codes 

Lattice codes are codes with a linear structure over the reals which are often used as a complexity-reducing code 
for an AWGN channel. A lattice code is an appropriately chosen subset of a lattice, which is a discrete subgroup 
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of IR n . We cannot simply use a lattice as a codebook on its own as it has infinite extent and, as a result, an infinite 
number of codewords, even for a finite dimension. Thus, for finite capacity channels, we limit ourselves to a subset 
of the lattice which is also often chosen to satisfy some type of transmit cost constraint (i.e. the power constraint 
on the AWGN channel). 

Definition 3: An n-dimensional lattice, A, is a set of points in R n such that if x, y G A, then x + y G A, and 
if x G A, then — x G A. A lattice can always be written in terms of a generator matrix G 6 l nxn : 

A = {x = zG : z G Z"} (5) 

where Z represents the integers. 

Definition 4: A lattice quantizer is a map, Q : W 1 — > A, that sends a point, x, to the nearest lattice point in 
Euclidean distance: 

x q = Q(x) = argmin llx - II | 2 (6) 
leA 

Definition 5: Let [x] mod A = x — Q(x). The mod A operation satisfies: 

[[x] mod A + y] mod A = [x + y] mod A Vx, y G R" (7) 
Definition 6: The fundamental Voronoi region, Vq, of a lattice, is the set of all points that are closest to the zero 

vector: V = {x : Q(x) = 0}. 

Definition 7: An (n, R) lattice code, C, is a code with elements taken from the intersection of some re-dimensional 

lattice A and a convex re-dimensional shape T (which is usually chosen to meet some type of power constraint.) 

C = A n T (8) 

\C\ = 2 nR (9) 
Remark 1: The constraint on a lattice code is often chosen to be the Voronoi region of another lattice that is 
nested within the original. For instance, let Ai C A 2 and let Vo,i be the Voronoi region of Ai. Then, a lattice code 
C can be defined as 

C = A 2 n V ,i = {x : x = y mod Ai, y € A 2 }. (10) 

For more on nested lattice codes and their applications, see [28]. Nested lattices are often employed to enable 
lattice decoding, where the receiver decodes to the nearest lattice point whether or not it is part of the allowable 
set of the lattice code. 

More broadly, we can consider nonlinear structured codes as well. These codes may prove quite useful in proving 
their own network capacity theorems, which may even be unavailable to linearly structured codes. However, such 
codes are beyond the scope of this paper. On a related note, in the network coding literature, linear codes have 
been shown to attain the same performance as random coding in the multicast case [29], [30]. However, Dougherty, 
Freiling, and Zeger constructed a (nonmulticast) network for which the linear network coding capacity is significantly 
lower than the non-linear network coding capacity [31]. 

A great deal of work has gone into showing that linear and lattice codes are sufficient for many channel coding 
and source coding problems. In the following section, we will briefly review some of these results. 

III. Structured Codes for Point-to-Point Problems 

We will now review some of the previous work on the existence of capacity-achieving structured codes for 
classical point-to-point problems. 

For additive noise channels, structured codes can achieve rates all the way up to capacity. Work in this area 
began with Elias' proof that there exist binary linear codes which are good for channel coding over the binary 
symmetric channel (BSC) [32]. Essentially, the proof shows that a codebook generated from a binary matrix 
with i.i.d. Bernoulli(i) entries has pairwise independent codewords that look as if they were drawn elementwise 
i.i.d. from a Bernoulli(i) distribution. As the standard random coding achievability proof only requires pairwise 
independence for the union bound, we can complete the capacity theorem. (See [33, §6.2] for a full proof.) 

In fact, Elias' method extends to a much larger class of channels. If we assume the alphabet is over a finite field 
and the uniform input distribution is capacity-achieving (which is the case for additive noise), then a matrix with 
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elements chosen i.i.d. and uniformly from the finite field will suffice. This is captured in the following lemma from 
Problem 2.1.11 in [34]. 

Lemma 1: Let w G ¥ k be the message and let the channel output be given by y = x + z where x, y, z 6 F n and 
z is an i.i.d. sequence. Then the capacity of the channel is given by C = log |F| — H(Z) and can be achieved with 
a linear code Q kxn so that x = wG. Specifically, for any e > and n large enough, Pr(c(y) / w) < e where 
c(-) is the maximum-likelihood (ML) estimate of w. Note that k < nC is required to stay below the capacity. 

Linear codes can also be used for compressing any discrete alphabet source so long as the rate is higher than 
the source entropy. Moreover, they can reach any point in the Slepian-Wolf rate region for distributed compression 
[35]. 

For additive white Gaussian noise (AWGN) channels, showing that lattice codes are sufficient to reach capacity 
was considerably more challenging. An AWGN point-to-point channel has an output Fel which can be written 
as Y = X + Z where X is the channel input and Z is i.i.d. Gaussian noise with variance N. Unlike the discrete 
alphabet case, AWGN channel encoders are usually subject to a power constraint of the form ^ Y^i=i ( X W) 2 — 
The capacity of an AWGN channel is well-known to be: 

c=itog(i + £) <ii) 

Much effort was focused on finding lattices that when intersected with an n-dimensional ball of radius \fnP 
centered at form a capacity-achieving code. Urbanke and Rimoldi showed that such lattices exist for minimum 
angle decoding [36]. Further work by Erez and Zamir showed that nested lattice codes can be capacity-achieving 
under lattice decoding, i.e. decoding to the closest lattice point whether or not it is an allowable codeword [37]. 

As in the discrete case, Gaussian source coding can be performed optimally with lattices [38]. Recall that the 
rate distortion function for mean-squared error for an i.i.d. Gaussian source with variance cr| is: 

= I log (12) 

Erez, Litsyn, and Zamir proved that there exist lattices which are simultaneously good for AWGN channel coding 
and Gaussian source coding [39]. These will be very useful to us in proving our theorems. Below we give a formal 
statement on the existence of lattice codes for channel coding and source coding: 

Lemma 2: Let s be a length-n Gaussian vector with variance cr|. There is a sequence of lattices, A n , such 

that quantizing s to the lattice points inside A n n Bq{^ n<r 2 s ) is sufficient for recovery at mean-squared error (or 
distortion) D. Furthermore, the number of points in A n n Bo(^Jno^) satisfies: 

1 



lim — log 

n^oo n 



A n DB ( J na 



r2- 



2 log d3) 



Transmitting points from the same sequence of lattices (up to inflation) intersected with a power sphere Bo(y/nP) 
followed by maximum-likelihood decoding is sufficient for achieving any rate below the capacity of an AWGN 
channel: 

lim - log A n n B {V^P) = \ log (1 + ^ ) (14) 

where P is the transmit power constraint, N is the noise variance, and the average probability of error goes to 
as n — > oo. 

See [39] for a full proof of the existence of such lattices. Note that their goodness for AWGN capacity under ML 
decoding essentially follows from Theorem 1 in [40]. There Loeliger shows that lattices from "Construction A" 
(the employed method of randomly generating lattices) satisfy the Minkowski-Hlawka theorem in expectation. This 
is all that is needed to invoke the results of Urbanke and Rimoldi for AWGN capacity under ML decoding [36]. 
In fact, one can establish that the same lattice can be simultaneously good for source coding and channel coding 
with lattice decoding [37]. Many classical multiterminal problems with a random coding solution also have a lattice 
coding solution. We refer the interested reader to [28] for an excellent survey of these ideas. 
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IV. Distributed Computation 

When we are interested in computing a function of distributed sources, structured codes can be extremely useful. 
In essence, an appropriately chosen structured code will commute with respect to the desired function. The function 
can then be applied to codewords instead of the original sources. This technique can reduce the required rates as 
perfect reconstruction of the original sources is no longer required. 

We first review the Korner-Marton problem for recovering the parity of two distributed, correlated binary sources 
[5]. Random binning fails in expectation whereas random linear binning succeeds. Next, we will review results 
that show that large gains are possible for computing functions of sources over noisy multiple-access channels [8]. 

A. Korner-Marton Problem 



Si 



Ri 



R2 



V 



u 



Fig. 1. Korner-Marton Problem 

Let the vector source (Si, S 2 ) be generated i.i.d. from the following joint probability distribution function (pdf): 



Pr(Si 
Pr(Si 



0,S 2 

o,s 2 



0) 

1) 



Pr(Si 
Pr(Si 



1,S 2 
1,S 2 



1) = 
0) = 



p 



(15) 



A simple calculation will show that Si and S 2 have uniform marginal distributions. We would like to reconstruct 
the mod-2 sum, U = Si © S 2 , at the decoder with vanishing probability of error. Note that H(U) = hs(p) where 
Hb{p) is the usual binary entropy function: 

Mp) = -plogp- (1 -p)log(l -p) (16) 

More formally, we would like to find the set of rates R\ and i? 2 such that there exist two encoders and a decoder: 

Sj : {0, l} n - {0, 1}"^ J = 1,2 (17) 

V : {0, l} nRl x {0, l} nR2 ->■ {0, l} n (18) 
such that the probability of error for recovering U goes to in the blocklength: 

u = D(fi(si),f 2 (s2)) (19) 



lim P (u / u) = 

n— >oo 

1 ) Optimal Rate Region: 

Theorem 1 (Korner-Marton): The rate region for distributed compression of U 
following constraints: 



Si 



(20) 

S 2 is given by the 



Ri > h B {p) (21) 
R2 > h B (p). (22) 
Proof: (Achiev ability?) Choose a linear source code, G G {0,l} nxnR with rate R > hsip) that is sufficient 
for losslessly compressing U. Have each encoder apply this code to its observed source to get wi = SiG and 
w 2 = s 2 G. These codewords are sent to the decoder which computes wi © w 2 = SiG © s 2 G = uG. Since G 
was chosen for recovering U, decoding is successful. 
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(Converse.) Consider the relaxation where the decoder has full knowledge of 52 and we would like to jointly 
encode S± and U to losslessly reconstruct U at the decoder. Note that any scheme that accomplishes this also gives 
the decoder a lossless reconstruction of Si. Thus, it can be shown that for joint encoding, R > H(Si, U\S2) = 
H(U\S2) = H(U) = hsip) is required for a vanishing probability of error. This implies that for separate encoding 
of Si and U, Ri + Ru > Iib{p). Similarly, we can get that R2 + Ru > /&b(p)- Setting Ru = gives the desired 
result. □ 

2) Performance of Best Known Random Code: For random binning with rates satisfying: 

Ri > h B (p) (23) 
R2 > h B (p) (24) 
Ri + R 2 > 1 + Mp) (25) 
it is easy to show that the decoder can reconstruct the source vectors si and S2 and u follows by taking the mod-2 
sum. We now argue that with random binning the mod-2 sum cannot be recovered at smaller rates. Suppose that 
Ri + R2 < 1 + hsip). We can correctly decode the sum with high probability if all typical pairs assigned to a 
particular pair bin indices have the same mod-2 sum. This ensures that the decoder will not get confused between 
several possible sums. There are approximately 2 n ^ 1+flB ^ typical pairs but there are at most 2 n pairs with the 
same mod-2 sum (even including atypical sequences). Thus, two typical pairs assigned to the same bin indices 
only have the same mod-2 sum with vanishing probability. As Ri + R2 < 1 + Hb(p), we will definitely have many 
typical pairs assigned to the same bins and these will almost certainly have different mod-2 sums. As a result, we 
cannot recover the mod-2 sum correctly at the decoder. 

It is possible that something more clever than random binning other than linear coding will lead to a successful 
proof. However, we are not currently aware of any other successful proof techniques. 

The Korner-Marton problem demonstrates that there exist problems for which purely random coding arguments 
are insufficient. However, the gains depend on the source correlations; for independent sources, there is no advantage 
to linear codes. A similar phenomenon has been discovered for correlated Gaussian sources by Krithivasan and 
Pradhan [11]. There if the sources are positively correlated and we only demand the difference at the decoder then 
lattice coding can be helpful. We now examine a problem for which structured random codes give large gains 
regardless of the underlying source dependencies. 

B. Computation over Multiple-Access Channels 

In the standard multiple-access problem, the decoder must recover the messages sent by each encoder. The rate 
region for this problem was established in [1], [2]. Suppose now that we are only interested in recovering a function 
of the transmitted messages. If the multiple-access channel (MAC) is simply a deterministic function of its inputs, 
then clearly it can be used as a reliable computational unit for evaluating that function. However, if noise is also 
injected, then some form of coding is required to compute reliably. 

In [8], we gave a class of strategies for computing functions over noisy MACs. As it turns out, structured codes 
are an essential part of the code construction. We briefly describe two distributed linear computation problems below 
and refer the interested reader to [8] for a more comprehensive study. The proofs of both results are reproduced 
here for completeness. 

1 ) Discrete Case: First we consider sending linear functions over a discrete linear MAC. 

Let F be a finite field and let the vector source [Si, S2, ■ ■ ■ , Sm) £ ¥ M be generated i.i.d. from some joint pdf. 

We would like to reconstruct the linear function, U = a\Si + 0:2 £2 H + olmSm, at the decoder with vanishing 

probability of error. Each source is seen by an encoder with channel input Xj G F for j = 1, 2, . . . , M. The channel 
output of a discrete linear MAC is given by 

Y = P1X1 + f3 2 X 2 + ... + (3 M X M + Z (26) 

where Z G F is drawn i.i.d. according to some pdf. 

More formally, we would like to find the highest computation rate R such that there exist M encoders and a 
decoder: 

Sj : ¥ nR -> ¥ n j = 1, 2, . . . , M (27) 
D : F n -f ¥ nR (28) 
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such that the probability of error for recovering U goes to in the blocklength: 



u = P(y)) 
lim P(u/u) 







Theorem 2: The capacity for computing U over a discrete linear MAC is given by: 

log |F| - H(Z) 



C = 



H(U) 



(29) 
(30) 



(31) 



In fact, a similar result holds for transmitting several (possibly correlated) linear functions. See [8] for more details. 

Proof: (Achiex 'ability.) Choose a matrix H that is appropriate for compressing U. Similarly, choose a good point- 
to-point channel coding matrix G for overcoming noise Z. At each encoder we set xj = [3J 1 a jSjHG. After the 
linear operation performed by the channel, we get: 



y = /?ixi + /? 2 x 2 H h /3a/x m + z 

y = uHG + z 



(32) 
(33) 



Due to G we can recover from the additive noise and due to H we can recover u with a vanishing probability of 
error. 

(Converse). For this class of MACs, we can simply allow the encoders to completely collaborate and get a tight 
upper bound. This reduces our problem to a point-to-point problem and we can invoke the separation theorem to 
get a converse: RH(U) < log |F| - H{Z). □ 

2) Gaussian Case: The natural extension of the discrete problem considered above to the continuous case is 
transmitting the sum of Gaussian sources over a Gaussian MAC at the minimal mean-squared error. When the 
source and channel bandwidths are equal, uncoded transmission is optimal. However, given more channel uses than 
source symbols, we would like to continue to use the additive property of the MAC to our advantage. Below we 
give an achievable scheme and a lower bound for refining the sum over many channel uses. 



S 2 ^ &. 
Sm — * 

Fig. 2. Reliable Addition over a Gaussian MAC 




V —17 



Each encoder, £j, sees an i.i.d. Gaussian sequence {Sj[i]}^ =1 with mean and variance cr|. For every k source 



symbols, we are allotted n = £k channel uses where £ e Z_| 



The encoders must satisfy average power constraints: 

1 11 

-5>^]) 2 <P VjG{l,2,...,M} 

i=i 

The channel output is just the sum of the channel inputs plus independent Gaussian noise: 

M 
3=1 

where {^[i]}f =1 is an i.i.d. Gaussian sequence with mean and variance N. 



(34) 



(35) 



(36) 
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Our goal is to reconstruct the sum of the sources, U = S± + S2 + ■ ■ ■ + Sm, at the decoder with the lowest 
possible distortion. Distortion is measured by the usual mean-squared error criterion: 



k 



(37) 



Theorem 3: Choose 5 > 0. For k large enough, the following distortion is achievable for sending k sums of 
independent Gaussian sources over a Gaussian MAC with n = Ik, t G Z + channel uses: 

i-\ 



Do = Mai 



N 



N + MP 

Furthermore, the distortion can be lower bounded by: 

D t > Ma% ( 



MN Y 



N + MP 



+ 5. 



(38) 



N 



, . (39) 
N + MP J 

Proof: (Achievability .) In [41], Kochman and Zamir develop an elegant joint source-channel lattice scheme for 
sending a Wyner-Ziv Gaussian source over a dirty paper channel. Our distributed refinement scheme consists of two 
main steps. First, we use uncoded transmission to send a noisy sum to the decoder. Then, we have each encoder run 
a version of the Kochman-Zamir scheme targeted at the desired sum, U. Assume 1 = 2. We thus have 2k channel 
uses to convey k sums. We will use the first k channel uses for an uncoded transmission phase. The decoder will 
then form an MMSE estimate U of the sum U = S\ + • • • + Sm and use this as side information for the next 
phase. Thus, U = Q + U where Q is an i.i.d. Gaussian sequence with mean and variance Ma 2 s N + MP - 

Choose a sequence of good lattices, A&, using [39] and scale them such that the normalized second moment of 
the lattice is MP. Let di, d2, . . . , &m be independent dither vectors drawn uniformly over the fundamental Voronoi 
Unif(Vo,fe), and made available to the encoders and decoder. 



region, dj ~ 

Each encoder transmits x, 



m y 3 



v ? and the decoder receives y: 



Vj = [jSj + dj] mod A fc . 

M 



(40) 



1 



E 



x ? - + z. 



The decoder then computes: 




M 



v , =1 j = i 
M 

x.j + az + 7 



mod Afc 



mod Afc. 



If the second moment of the term inside the modulo operation does not exceed MP, the second moment of the 
lattice, then we can guarantee that: 



lim Pr 

k^oo 



a 



M 



1 



M 

E 



x~ + az + 7q 



1. 



(41) 



See [37] for a detailed discussion of the effect of the dither in this step. The second moment can be controlled by 
requiring that: 



a 



1 ) {M Z P) + a z N + 7V5 < MP. 



(42) 
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This equation will be satisfied by our final choice of the constants a and 7. The decoder's estimate of the sum is 
given by: 

u = (3r + u 




(l-/3 7 )q + u. 



This estimate gives the following mean-squared error: 




D = B 2 \ ( —j= — 1 ) M 2 P + a 2 N + (1 - 7 ) V 



Q- 



We define the following constants: 



MPVM 

a 



MP + N 



MP ( MN 



and let 7 — > 70 from below as k — > 00. This ensures that Equation (1421 ) is always satisfied. We also set: 

°o7 
1 MP 

As k — » 00, we get that the achieved distortion is: 

o iV MiV 

D = Mai . (43) 

6 N + MP iV + MP v ' 

This proves the theorem for 1 = 2. For all higher values of I, the scheme can be repeated with the final estimate 
from the last refinement taken as side information for the next stage. Note that since this works in expectation over 
the dither random vectors then there must exist fixed vectors which can be used to achieve the same distortion. 

(Lower Bound.) Using steps from the converse to the multiple-access problem (see [42, pp. 399-407]) as well 
as the independence of the sources, we can get that I{X\,Xi, ■ ■ ■ , Xm', Y) < \ log (l + Hf-)- It is also clear that 
the rate distortion function for jointly compressing the sum is given by Rjj(D) = ~ log ^75^- By applying the 
data processing inequality, we get the desired lower bound. □ 

The significant gap between the lower bound and the achievable scheme seems due to the distributed nature of 
the problem. The encoders quantize each source prior to each transmission whereas the lower bound allows for 
cooperation. Therefore, it seems that the lower bound is quite loose for large I. We presented a slightly improved 
achievable scheme as part of [23], [43] but there was an error in the proof. 

3) Performance of Best Known Random Code: For the problems considered above, random coding arguments 
perform quite poorly. As in the Korner-Marton problem, the best performance one can hope for is complete 
reconstruction of the sources at the decoder followed by computing the sum. This reduces the rate at which 
functions are computed proportionally to the number of users if the sources are independent. 

For the discrete problem considered in Section IIV-B.1I the random coding argument is as follows. Each en- 
coder generates 2 nR codewords according to the uniform distribution on F (which is also the capacity-achieving 
distribution). Typical sequences are mapped to these codewords. Note that nearly all the probability is concen- 
trated in approximately 2 nH ( Sl < S2 >—> SM > M-tuples of jointly typical sequences. Of these sequences, approximately 
2nH(s 1 ,s 2 ,--;S M \U) M^pi^ h ave the same desired function. For a joint typicality decoder, the probability that an 
incorrect set of vectors is found to be jointly typical with the channel output is approximately 2~~ nI ( Xl > X2 '"' > x m> y ). 
Note that I(Xi,X 2 , ... , X M ;Y) = log |F| - H(Z) for the chosen input distribution. Since H(S\, S 2 , ■ ■ ■ , Sm) > 
H(S±, 52, ... , Sm\U), two sets of M-tuples have the same desired function with vanishing probability. Thus, an 
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incorrect function will be jointly typical with the channel output unless we choose R such that we can distinguish 
all typical M-tuples: 

R < ^\-H(Z) (44) 
H(S\, 52, ... , Sm) 

This is equivalent to requiring complete reconstruction of the sources. 

For the Gaussian problem, a similar argument shows that random coding arguments can only achieve the following 
distortion: 

2 / N \ £ / M 

D = Mai — — (45) 

b \N + MP J 

Again, this is equivalent to reconstructing each source at the decoder using a standard multiple-access technique 
and then adding them up. 

Using structured codes for distributed computation allows several users to exploit the natural function of the 
channel while overcoming the noise. With a random coding proof, one can only overcome the noise if all of the 
sources are completely reconstructed at the decoder. As the problem statement includes the evaluation of a function, 
it seems natural that the codebooks should be structured with respect to the function. It is not so clear that such a 
property is desirable when we only want to send messages across a network. The problem considered in the next 
section implies that structured codes have a useful role to play in transmitting bits as well. 



V. Relay-Type Problem 

So far we have seen that structured random codes offer gains when we are interested in computing a function 
in a distributed fashion. We now give a multiterminal channel coding scenario for which structured random codes 
are better suited to the problem than purely random codes, even though we are only interested in reproducing bits 
at the decoder. 



Zi 




-1 z 2 



Fig. 3. Sum-Difference Relay Channel 

Two separate users would like to send independent messages w±,W2 £ {1,2, .. . ,2 nR } to a decoder across a 
channel with the help of relays (see Figure [3]). Encoder 1 produces channel input X% G M with average power 
constraint P and Encoder 2 must produce channel input X 2 € R with the same constraint. One relay terminal 
sees the sum of these signals plus noise: Y\ = X\ + X 2 + Z\. The other terminal sees the difference plus noise: 
Y 2 = X\ — X 2 + Z 2 . Z\ and Z 2 are independent and Gaussian with variance N. Finally each relay has a noiseless 
bit pipe to the decoder with rate Rq. We would like to determine the maximum achievable symmetric rate R to 
the decoder for a given value of Rq. 

More formally, we would like to find the maximum R for which there exist four encoders and a decoder: 

£j : {0, l} nR -> R n j = 1, 2 (46) 
£ t : R n -» {0, l} nRa £ = 3,4 (47) 
V : {0, l} nRo x {0, l} nRo -» {0, l} nR x {0, l} nR 
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such that the probability of error for recovering w\ and W2 goes to in the blocklength: 

(wi,w 2 )=V(y 3 ,y 4 ) (48) 
lim P((wi,w 2 ) + (w u w 2 )) = (49) 

n^oo 

Note that, ideally, the decoder would have full access to the sum and difference seen at the relays. It could 
then add and subtract these to get the original signals without interference. Our lattice strategy is geared towards 
decoding only the sum and difference of the messages at the relays and passing these functions along to the decoder. 
As seen earlier, implementing such a scheme with random codes is not possible as we effectively need to send the 
messages in their entirety to get the sum. 



A. Lattice Symmetric Rate 

We will use lattices to allow each relay to "compute-and-forward" either the sum or the difference to the decoder. 
We will proceed by using the scheme for computing the sum of Gaussian sources presented in Section IIV-B.2I and 
then connecting the resulting distortions to transmitting bits. The following lemma will be useful in our analysis. 

Lemma 3: Assume, for the problem described above, k large enough, and n = ki, I G Z + channel uses, that 
each user can transmit an i.i.d. Gaussian sequence of length-A; with mean and variance er| (si and S2) such that 
the decoder can make estimates §1 and §2, each with mean-squared error (or distortion) Dg^. Then we can design 

encoders Ej, j = 1, 2, 3, 4 and a decoder V for transmitting at any rate less than ^ log fzjf^J from each encoder 
for any average probability of error greater than zero. 
See Appendix A for a full proof. 

Theorem 4: For the two relay problem described above, the following symmetric rate is achievable using lattice 
codes: 

R LAT = min (\ log f ~ + £ J , Rq] . (50) 



2 \2 iV, 

Proof: Choose Sx and s 2 to be i.i.d. length-A; Gaussian vectors with mean and variance cr|. We will use 
the channel network n = kl times to transmit the sources. Use Theorem [3] to pick a code for sending the sum 

u = si + S2 to Encoder 3 at distortion D = ct| ( N 2 + 2 p ) ■ Due to the symmetry of the underlying lattice code and 
the negative sign on the lower path, Encoder 4 will be able to reconstruct the difference v = si — S2 at distortion 
D as well. 

In order to send the sum and the difference to the final decoder, we will need to requantize them. Pick a Gaussian 
source code for compressing a variance 2a% source to distortion Dq = 2ag2~ 2£Ro . By the triangle inequality, this 
requantization step will cause the total distortion for u and v to be at most Dq + D. At the decoder we estimate 
si by §1 = |(u + v) and S2 by §2 = |(u — v). It can be checked that these give us the original sources to within 

distortion max ^4cr| ^ n^p ) > 2^0^ ■ Finally, we calculate the rate achievable by each encoder using Lemma [3j 



R LA7 = Yl 



4 



= xmnQlogQ + ^),2Jb)-llog(4). (51) 

Thus, for I large enough, we can achieve the desired rates. □ 

The number of refinements, I, plays an important role in the proof above. Essentially, the sum and difference are 
"too large" to be sent to the decoder without requantizing them. This requantization doubles the end-to-distortion 
but since we refine the sum and difference many times, the effect of this doubling is negligible. 

We can also achieve the desired rates using a nested lattice code where the relays decode the sum and the 
difference modulo the coarse lattice. For an example of this type of scheme, see [19] for an application to a 
two-way relay channel. 
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A purely random code will not have access to the capacity as we cannot decode the sum and difference without 
decoding the sources in their entirety. Below we describe the best known random coding strategy for such a problem 
and demonstrate that it is indeed below the capacity. 



B. Performance of Best Known Random Code 

For this type of problem, the best known strategies are decode-and-forward and compress-and-forward. For 
decode-and-forward, the relays recover the message intended for the decoder and pass it along. For compress- 
and-forward, the relays quantize their observed signals according to Gaussian codebooks and forward these to the 
decoder. 

Theorem 5: The best achievable rates for decode-and-forward and compress-and-forward with Gaussian code- 
books are given below: 



R n 



mm 



1 



log 



2P\ 
1+ iV 



Rc 



l 



1 / 2P(2 2R ° 

2 g l 1+ 2P + N2 2 *° 



(52) 



(53) 



Proof: For the decode-and-forward scheme, the message from Encoder 1 is meant to be decoded at Encoder 3 
and the message from Encoder 2 is meant to be decoded at Encoder 4. However, as Encoders 3 and 4 see the 
outputs of statistically equivalent multiple-access channels, they can always decode the message intended for the 
other terminal as well. Thus, we can use the standard multiple-access result to bound the rates and we get that 
the maximum symmetric rate is 4 log (l + ^) (assuming the relay links to the decoder are not the bottleneck). 
For the compress-and-forward scheme, we can think of quantizing the signals as simply adding additional noise to 
the observed signals. We can show that this additional noise has variance N = §S%zri ■ Adding the two received 



signals together gives an estimate of the first transmitted message with SNR 
a similar estimate of the second user's message. □ 



— 2 2R o 
AP 

2N+2N 



. Taking the difference gives 




Upper Bound R n 

Lattice Code 

Decode-and-Forward 

o Compress-and-Forward 



SNR (P/N) 
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Fig. 4. Achievable Symmetric Rate Points for Sum-Difference Relay Channel 



As an example, we let Ro = \ log (l + -£) and we evaluate the performance of each scheme in Figure |4] The 
lattice coding scheme is superior to both schemes for SNRs higher than 1.5. 

The decode-and-forward scheme completely ignores the benefits of using the sum and the difference and suffers 
as a result. Although the compress-and-forward scheme lets the decoder see a version of the sum and the difference, 
this version also includes the noise as we naively forwarded the resulting signal. (Note that if Rq tends to infinity, 
compress-and-forward is optimal.) By using a lattice, we can just "compute-and-forward" the desired functions. 

Most work in multiple-access theory focuses on how many bits can be sent from a set of users to a single 
receivers. This example indicates that for AWGN networks it may also be useful to characterize exactly how many 
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linear functions can be sent per channel use. This question is partially answered by Theorem [3] In the next section, 
we will see that converting an AWGN network into a system of linear equations can be fruitful. 

VI. Network Coding with MACs 

Recently, there has been a great deal of interest in network coding [29], [30], [44]. The key idea is that for 
multicasting from one sender to multiple receivers over a network of point-to-point channels, the encoders should 
sometimes only send a function of their incoming messages (i.e. routing is suboptimal). If the network now includes 
MACs, it is clear that we should try to use these MACs to compute functions of incoming messages when required. 

As seen in the previous section, it may be useful to think about multiple-access links in terms of how many 
functions can be reliably transmitted across them instead of just the standard multiple-access rate region. As detailed 
in Section |IVJ structured codes are a natural way of constructing codes for distributed computation. 

A channel network is usually thought of as a graph where the vertices are the encoders and decoders and the 
edges are the point-to-point channels with known capacities. For our problem, we will need a bit more notation to 
cleanly represent both point-to-point channels and MACs. 

Definition 8: A multiple-access network, G M ac, consists of the following elements: 

1) Vn- the encoder/decoder nodes of the network. Each node, v, has a unique label taken from the positive 
integers, v G Z + , and consists of a decoding function g v v for each incoming edge (vj,v) and an encoding 
function f VVk for each outgoing edge (v,Vk)- 

2) v s : the source node. One element of Vat. The source transmits the message, w € {1,2, ... , 2 nR }. 

3) (v R , vf, . . . , v R ): the receiver nodes. Each one is an element of Vn and produces an estimate of the transmitted 
message, ui£. 

4) V M A<> the MACs in the network. Each MAC, m, has a unique integer label, m G Z + . Each MAC has a noise 
variance N m G R + . 

5) £ nn- the directed point-to-point channels in the network. Each channel has a unique integer label, e^N G Z + , 
and the labels of its inputs and output nodes are given by the functions v m (cjviv) and v 0VT (eNN) respectively. 
The noise variance for the channel is given by N e G R + . 

6) £nm- the input edges from nodes to MACs. Each edge has a unique integer label, e^M G Z + , and the labels 
of its inputting node and destination MAC are given by the functions v m (eNu) and v 0UT {eNM) respectively. 

7) £mn- the output edges from a MAC to a node. We assume that the output of a given MAC is only observed 
by a single node. Each edge has a unique integer label, Cmn G Z + , and the label of its MAC and destination 
node are given by the functions v m (eMN) and v 0UT (eMN) respectively. 

8) X v . Vh [i]: the channel input on the edge (vj,Vk) at time i. The encoders are constrained to only produce 
channel inputs from time i = 1 to time i = n. 

9) Yy.^jji]: the channel output on the edge (vj,Vk) at time i. 

We also assume that there are a finite number of nodes and channels in the network, |Vjv| + |V M ac| + \£nn\ + 
\£nm\ + \£mn\ < °o. 

Definition 9: A multicast rate, R, is achievable if Ve > and n large enough there exist encoding and decoding 
functions for the network such that the average probability of error is less than e: 

w e = f v n(Y%) 

Pr ({wi / w} U • • • U {w L / w}) < e, (54) 

where w G {1, 2, . . . , 2 nR \ and Y\ represents all the channel outputs observed by the £ th receiver. 
Definition 10: The multicast capacity is the supremum of all achievable multicast rates. 

Definition 11: A point-to-point network, Q ?0 i m = (Vn,£nn), is just a multiple-access network without any 
multiple-access nodes, V M ac = £nm = £mn = 0- 

Definition 12: A unit bit pipe network, Q mE = (V,£), is just a point-to-point network except all of the channels, 
£, are taken to be noiseless bit pipes with unit capacity. The encoding/decoding nodes are given by the set V. 

Our scheme will give achievable rates for multiple-access networks comprised of either discrete linear or Gaussian 
MACs. We express the achievable rate through a new point-to-point network that results from an appropriate 
transformation of our original network. The achievable rate is then given by the multicast capacity of the point-to- 
point network. We will also demonstrate that in some cases our achievable rates coincide with the simple upper 
bound due to the max-flow min-cut theorem of Ford and Fulkerson [45]. 
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We now briefly review some results for multicasting over point-to-point channel networks. In [44], it was shown 
that for a unit bit pipe network the multicast capacity is given by the max-flow min-cut theorem. For each receiver, 
calculate the maximum information flow across all cuts that separate the source node from that receiver. The 
multicast capacity is the minimum of all these max-flow values taken over all cuts and receivers. In [29] and [30], 
it was shown that linear encoding and decoding over a finite field is sufficient to achieve the multicast capacity. 
Bounds are also given on the required field size. It was independently and concurrently shown by Ho et al. in [46], 
Jaggi et al. in [47], and Sanders et al. [48] that the field size only needs to be larger than the number of receivers. 
We reproduce the version from [46] below as it will be useful to us in proving our main theorems. 

Definition 13: Let Q PlfE = (V, E) and let ¥ q be a finite field of size q. An algebraic network code is a set of linear 
functions for a unit bit pipe network. Specifically, the encoding function from node Vj to node is constrained to 
be a linear function of its observations from each incoming edge: 

X VjV k [i] = ^ a VrVs Y VrVj [«]• (55) 

where i^uji] is the value seen by node Vj at time i on the incoming edge from node v r and Y VrV .[i],a VrVj G ¥ q 
for all v r G V. 

Lemma 4 (Ho et al): Let Q = (V,£) be a unit bit pipe network with a single source and L receivers. The 
multicast capacity is given by the max-flow min-cut bound and can be achieved by an algebraic network code over 
any finite field larger than L (¥ q , q > L). 

For a full proof, see [49]. 

Now we need to map these linear functions onto the multiple-access channels in the network. We first explore 
two examples that are a variant of the butterfly network given in [44]. Then, we state our theorems on multicasting 
over general finite field and AWGN multiple-access networks. 

A. Motivating Examples 

We now look at two variants of the butterfly network from [44, Figure 7] to demonstrate our coding idea. In the 
original butterfly, recall that it is optimal to send one bit down the left path, a different bit down the right path, and 
the mod-2 sum of these bits down the center path. In our examples, the center path now includes a multiple-access 
channel. Our goal will be to add together two bitstreams on the MAC. The first example examines at the binary 
alphabet case and the second looks at the Gaussian case. 
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Fig. 5. (a) Binary multiple-access variant of the butterfly network. The MAC in the center is a noisy mod-2 adder with i.i.d. Bernoulli(p) 
noise Z (b) Using a linear code, we can achieve the multicast capacity of the MAC butterfly network which is equivalent to the multicast 
capacity of this transformed network. Here, R LF — 1 — Iib{p). (c) With a decode-and-forward strategy, we can only achieve rates on the 
original network that are achievable on this network. Here, i? DF = |(1 — }ib(jp)) 

1 ) Mod-2 Adder MAC: Consider the channel network in Figure [6£a). Each vertex on the graph represents a 
decoder/encoder pair. The sender is at the top of the graph and the two receivers are at the bottom. The labeled 
edges represent noiseless bit pipes each with capacity C. At the center of the graph is a MAC with inputs X\ (from 
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the left) and X2 (from the right) and output Y = X\ © X2 © Z where Z is an i.i.d. Bernoulli(p) sequence. Note 
that the sum rate of this MAC is upper bounded by 1 — h B (p). 

Theorem 6: For the channel graph from Figure [5] (a) the multicast capacity is: 

R = C + min(C,l - h B (p)) (56) 
Proof: (Converse.) Applying the cutset bound gives that the rate to each receiver is upper bounded by: 

R < C + min (C, 1 - h B (p)) (57) 

(Achievability.) We have a block b of n(C + min (C, 1 — h B (p))) bits to transmit to both receivers. We will break 
up b in two ways. For the first, we write b = [bnbi2] where the first chunk is of length nC and the second 
is of length n(min(C, 1 — h B (p))). For the second, we write b = [b2ib22] where the first chunk is of length 
n(min (C, 1 — h B (p))) and the second is of length nC. We transmit bn down the left path and b22 down the right 
path. From bn we automatically know b2i and from b22 we know bi2- We send the mod-2 sum u = b2i © bi2 
reliably across the MAC using the linear code from Theorem[2] Finally, this mod-2 sum is conveyed to the receivers. 
The left receiver can compute bi2 = u © b2i and the right receiver can compute b2i = u © bi2 to fully recover 
b. □ 

Standard random coding arguments cannot attain the optimal performance over the network in Figure [5] (a). The 
decode-and-forward and compress-and-forward rates are given by: 

i? DF = a + min(c,i-^M) (58) 

R CF = C + min (C(l - h B (p)), (1 - h B (p))) (59) 

If the capacity C of the point-to-point links is small enough, then fully decoding the input messages to the MAC 
is sufficient. If the capacity C is large enough, forwarding the output of the MAC is sufficient. However, in the 
intermediate regime, structured codes are necessary. 

2) Gaussian MAC: Consider the AWGN channel network in Figure [6] (a). Each vertex on the graph represents 
a decoder/encoder pair. The sender is at the top of the graph and the two receivers are at the bottom. All encoders 
must satisfy an average power constraint, - 2~27=l x ji^\ 2 — P- The Z m , m = 1, 2, ... ,7 are drawn i.i.d. according 
to a Gaussian distribution with mean and variance N. 

Again, it is convenient to consider sending Gaussian sources across the network and then link the performance 
back to sending bits reliably. 

Lemma 5: For a given Gaussian multiple-access network, assume for k large enough and n = k£, £ G Z + 
channel uses, that the sender can transmit B i.i.d. Gaussian sequences of length-/c with mean and variance a 2 s 
such that each receiver can make an estimate of each source sequence to within distortion D. Then we can design 
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encoders and decoders for multicasting at any rate less than || log I -fi 1 for any average probability of error greater 
than zero. 

See Appendix A for a full proof. 

Theorem 7: The following multicast rate is achievable on the channel network in Figure Ob) using a structured 
code: 

fi =M i+ £H iog G + £> <6o) 

Proof: Choose si and S2 to be i.i.d. length-A; Gaussian vectors with mean and variance <jg. We will use the 
channel network n = kl times to transmit the sources where I G Z + will be specified later. Use Theorem [3] to pick 

a code for sending the sum u = si + S2 across the MAC at distortion D = 2<r| ( jf^p ) ■ 

We transmit si down the left path and S2 down the right path at distortion D. This requires rate | log (| + 
We also transmit the sum of si and S2 down the MAC and requantize it to distortion 2D for transmission to the 
receivers. 

At the left receiver we have estimates u and §1 and we form an estimate S2 = u — §1. The left receiver knows 
the source vectors at distortions y and respectively. (The distortions for the right receiver follow by analogy.) 
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Using Lemma [5] and taking the worst distortion for each source, our achievable rate is given by: 




Note that we still have | log (l + ^) — | log + ^) rate on both the left and right paths. We use this to send a 
common message. Finally, we get that for I large enough, we can achieve the desired rate. □ 
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Fig. 6. (a) Gaussian multiple-access variant of the butterfly network, (b) Using a lattice-based code, we can achieve any rate on the MAC 
butterfly network that is achievable on this transformed network. Here, R]f — i log + where P is the per user power of the MAC. 

(c) With a decode-and-forward strategy, we can only achieve rates on this network where i?° F = | log ( 1 + jf- j 



As in the discrete case, random coding arguments will not suffice for attaining this performance. The best 
strategies are analogous to that developed in Theorem [5] The encoder following the MAC either decodes the 
messages in their entirety or quantizes the observed signal and forwards it. Below we give the best achievable rates 
for decode-and-forward and compress-and-forward with Gaussian codebooks: 

1 / P\ 1 / 2P\ 

^=2 l0 H 1+ ivJ + 4 l0 H 1 + ^J 

^ = ^ log ( 1 + ^) + ^ log ( 1 + ^(3PT^)) 

B. General Networks 

We now give two multicasting results for general multiple-access networks. First, we give the multicast capacity 
for when the MACs are constrained to be noisy linear functions over a finite field. Second, we give achievable 
rates for any Gaussian multiple-access network. 

1) Finite Field Multiple-Access Networks: We assume all of the MACs in our channel network Gmac are 
constrained to be of the form Y = a\X\ + 02^2 + • • • olkXk + Z where a; 6 F, \ {0}, Xi G ¥ q , and Z 
is an i.i.d. random variable on the alphabet ¥ q . 

Theorem 8: The max-flow min-cut bound for multicasting is achievable for a finite field multiple-access network, 
G, if q > L, where q is the MAC field size and L is the number of receivers in the network. 

By using Theorem |2j we can transform all the MACs in the network into reliable adders. They can then be made 
part of an overall network code chosen using Lemma |4] For a detailed proof, see [8]. 
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2) Gaussian Multiple-Access Networks: We now assume that all channels are either AWGN point-to-point 
channels or Gaussian MACs. We further assume each user faces an identical power constraint and that the channel 
quality is controlled by the noise variance N m £ 

More formally, all channels in our channel network Q M ac are constrained to be of the form Y m = X\ + Xi + 
. . . Xj + Z m where Z m is an i.i.d. Gaussian sequence of mean and variance N m . Furthermore, the Xj must 
satisfy power constraints of the form - Yli=i ( x j[*]) 2 < P- 

The naive approach to a Gaussian multiple-access network would be to first convert all channels, including 
MACs, into bit pipes using standard random coding arguments. Network coding could then be performed over the 
resulting bit pipes. This ignores the potential of the MACs to compute as part of an overall network code. Another 
approach would be to transmit uncoded and use the MACs as noisy linear adders. However, this allows noise to 
build up unnecessarily as our signals propagate to the receivers. Our strategy uses a lattice code to reliably compute 
linear functions over the MACs in the network. This effectively converts the MACs into linear processing units 
and reduces the network to a point-to-point network for which multicasting is well understood (see Figure [7] for 
an example). The key question is what rates should we assign to the incoming and outgoing links to these nodes 
in our reduction. We will call this the linear processing rate and we will demonstrate that it is at least: 
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where N m is the noise variance associated with that MAC and J m is the number of inputs to the MAC. 



0* 




Gaussian MAC Network 



rj->-™2 




WL 



1 



Structured 
Code 



-a 



R-2 

^ CHAD D A 

I — Ri\ ll; 



Ri 

i — \s Transformed 

Gaussian MAC Network 





WL 



Rn 



2 ^ V Jm ~ N m 

Fig. 7. With a structured code we can convert every MAC in a network into a reliable linear relay with incoming and outgoing rate links 
given by the linear processing rate. Note that J m is the number of inputs to the MAC (2 for the MACs on the left and 3 for the MAC on 
the right) and N m is the noise variance for MAC m. 



We now give a formal statement of our result. 
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Definition 14: Let Q MKC be a Gaussian multiple-access network. An equivalent point-to-point network, Q' = 
(V',£'), is constructed from the original network by the following steps: 

• Let the set of encoder/decoder nodes, V', in the new network be given by the original encoder/decoder nodes 
as well as the original MACs, V' = Vn U V MA c- 

• Let the channels in the new network, £', be given by the original point-to-point channels as well as the input 
and output edges to the MACs, £' = £nn U £nm U £mn- The connectivity of these edges is the same as in 
the original network. 

• Set the capacity of the edges taken from £nm and £mn to be the linear processing rate, Rlp- 

Theorem 9: Let Qmac be a Gaussian multiple-access network. Any multicast rate achievable on the equivalent 
point-to-point network is achievable on the original network for the following linear processing rate: 

1 f 1 P \ 

where N m is the noise variance of the MAC m associated to these edges in the original network and J m is the 
number of inputs to these MACs. 
See Appendix B for a full proof. 

Using a purely random code, we can also achieve the same network transformation but with a different linear 
processing rate given by: 

For sufficiently high SNR, the performance of the structured random code is superior. 



VII. Discussion and Open Problems 

In this paper, we have examined several interesting problem classes where random coding arguments seem 
insufficient. Surprisingly, for some of these problems, structured random codes are enough to close the gap and 
determine a tight lower bound to capacity. These structured codes were used to allow certain decoders in the network 
to recover a function of messages reliably. These functions could then be passed along to a final decoder which, 
equipped with a full rank set of linear equations, could decode the individual messages. Such a phenomenon has 
already been well-characterized in the field of network coding. However, in the standard network coding problem, 
messages are collected at an encoder and then their function is computed. For a graph of point-to-point channels, 
this is optimal but if the graph also includes multiple-access channels, structured codes seem necessary to convert 
these into reliable computational units as well. 

Our definition of a purely random code is certainly debatable. Still, it seems to us that any code that is capable 
of recovering these results has some form of structure, whether or not it is generated in the same manner as our 
codes. We close by listing some open problems around the topic of structured random codes and AWGN network 
capacity. 

Open Problem 1: All of the structured codes considered in this paper were linear. It would be interesting to find 
non-linear structured codes and a canonical problem class where they outperform purely random codes. 

Open Problem 2: A proof that if one achieves the maximum rate in one of the problems considered in this 
paper then almost all codewords of the underlying code belong to the same linear subspace. This seems most 
straightforward for the Korner-Marton problem or computation over MACs. 

Open Problem 3: A tight characterization of the best achievable distortion for Gaussian computation over MACs 
(see Theorem [3]). We currently suspect that the lower bound is quite loose for large I. 

Open Problem 4: Determine the multicast capacity for Gaussian multiple-access networks. 

Open Problem 5: Find a transform from an arbitrary AWGN network capacity problem (multiple senders and 
receivers) into a system of linear of equations and find achievable rates using lattices. Is this an optimal strategy 
in general? 



20 



Appendix I 
Achievable Rates from Distortions 

We now show that if we can transmit an i.i.d. Gaussian sequence from a transmitter to one or more receivers at 
a given mean-squared error then we can also communicate bits. 

Theorem 10: Let s be a length-A: i.i.d. Gaussian vector with mean zero and variance a% available at a transmitter. 
Assume that we have a coding scheme that uses n = k£ channel uses, k,£ G Z+, to send Sj to a subset G C 
{1,2,..., L} of the receivers with mean squared error at most D^. Then, there exists a coding scheme that can 

send a message w from the same transmitter to all receivers in G at any rate R < ^ log (^jyf-^J for any average 
probability of error greater than zero. 

Proof: Fix k and £. Choose the encoders and decoder in the network such that we achieve the specified distortions 
at the receivers. We have by the data processing inequality: 

! lo s(^) <i(s k ;S%) (66) 

<I(X n ;Y g n ). (67) 

Thus we know that there exists a multiletter input distribution, p(x n ), such that the mutual information to each 
receiver is lower bounded by the left-hand side of (l66l ). We now define supersymbols of length n: 

X[i] = [X[(i-\)n + l],X[(i-l)n + 2],--- ,X[in}} 
Y t [t\ = [Y e [{i-l)n + l],Y e [{i-l)n + 2],--- ,Y e [in}} . 

The supersymbols X and Y take values in the alphabets X = R n and y = W 1 respectively. 

Keep all encoders and decoders in the network the same as in the distortion-achieving case except for those 
at the source and the receivers, vg,Vg, \/g G G. Thus, p(y g \x) is a memoryless channel. Generate a random 
codebook with 2^ length N codewords (one for each message in {1, 2, ... , 2^}) with each symbol drawn i.i.d. 
from X for some N G Z + and R > 0. The g th receiver upon seeing Y-* uses a maximum likelihood rule to infer 
the original message w. Denote this estimate by w g . It follows from [50] that for such a channel and N large 
enough, the average probability of error over codebooks and messages for receiver g, Pr(Wp ^ W) can be made 
less than j- if R < I(X;Y). 

It follows from the union bound that the probability that any receiver is in error averaged over all codebooks 
and messages satisfies: 

Pe<Yl ^9 ^w)<e. (68) 

gdG 

Finally, we get that there exists at least one fixed codebook with average probability of error at most P e , otherwise 
the average over all codebooks would not hold. This completes the proof. □ 

Appendix II 
Gaussian Network Coding Proof 

In this appendix, we provide a full proof of our Gaussian multiple-access network coding theorem for complete- 
ness. First, we show how to compute a linear function of Gaussian sources over a Gaussian MAC at the desired 
linear processing rate. 

Lemma 6: Let Si, S2, ■ ■ ■ , Sj be Gaussian sources with variances af, cr|, . . . , a 2 j respectively. Let = maxj u|. 
Each source is seen at one encoder with power P which faces a Gaussian MAC with noise variance N. Let q be 
a positive prime number and let U = j3\S\ + /32S2 + • • • + PjSj where (3j G {0,1,2, ... ,q — 1}. Then given k 
vectors of source symbols and n channel uses where n = ki, the decoder can make an estimate of U at distortion 

/ TAT \t 

D e = J(q- l) 2 al AX i^—pj ■ (69) 

Proof: The bulk of the work is done by Theorem [3] To send a linear function, we choose a lattice, A, for use 
in refining a sum with Theorem [3] as if all the sources had variance (q — 1) 2 o"m AX - At each terminal, we quantize 
Tj = (3jSj + Wj onto the lattice A where Wj is an i.i.d. Gaussian random variable available as common randomness 
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to both the encoder and the decoder. Its variance is chosen such that the variance of Tj is matched to the design 
variance of A, crjy = (q — 1) 2 <7 MAX — Pj&j - The decoder makes an estimate of the sum T\ + T2 + • • • + Tj and 
removes the common randomness variables, Wj, j = 1, 2, . . . , J, to get an estimate of U at the desired distortion. 
Note that as this works in expectation over the Wj, there exist fixed constants w\,W2, - ■ ■ ,wj that can serve the 
same role. □ 

We are now ready to prove our main theorem. 

Proof of Theorem® Choose q to be a prime number such that q > L where L is the number of receivers. We will 
use the channel network to convey Gaussian sources of length k. We will then connect the distortion performance 
back to sending bits using Lemma [5] Construct a new point-to-point channel network using Q' = (V',£ r ) as in the 
statement of the theorem. Let C be the multicast capacity of the Q' which is given by the usual max-flow min-cut 
characterization. We would like to show that for any S, e > 0, we can achieve a multicast rate R = C — 5 on the 
original network with average probability of error not exceeding e. 

First, using Lemma [4] we find an algebraic network code for Q' that can be used to achieve a rate R = C' — f . 
This basically involves breaking every channel into several capacity A channels through time-division where A > 
is small enough that the rate loss due to rounding error is negligible. Thus, for a chunk of channel uses we get 
a A rate bit pipe which we designate to carry exactly one linear function. Let P\ , N\ > be chosen such that 
A = I log ^1 + -j^M . The number of inputs to a node is clearly upper bounded by the number of A bit pipes in the 
network which itself is upper bounded by: 

I Supper I = (max ^ J \S'\ (70) 

which is a constant that does not depend on n. 

Each node is thus sent a finite number of functions, f/f, If* , • • • , Uj. Assume that each of these have variance 
at most afj. It makes MMSE estimates of these and prepares a new function, V = fi-JJ\ + P2U2 + • • • + PjUj 
with Pj G {0, 1, . . . , q — 1} for each outgoing chunk of channel uses according to the algebraic network code. This 
function can be transmitted to the receiver with distortion at worst: 

D t = 1) 2 4 ( N "+ Px ) £ - (71) 

according to Lemma [6] Note that this holds whether the node in the new network is an actual node in the original 

network or a MAC. Thus, the processing at a node increases the distortion by at most a factor |£upper| {l — I) 2 - 
C — — 

Let 7 = — j- 3 - which can be made an integer by choosing A appropriately. At the source we will create 7 i.i.d. 
Gaussian sources S\ , , . . . of length k and variance 1. These will be relayed to the receivers by means of the 
algebraic network code and the coding method described above. The receiver will see functions of these original 
sources at some distortions. First note that the distortions will not exceed (|Vjv| + |Vmac|)|£upper|(<Z — I) 2 - This is 
just the number of processing nodes multiplied by the maximum increase factor due to processing at one node. 

The functions of the sources seen at the decoders can be written as a matrix transformation just as the original 
algebraic network code description. If the algebraic network code induces a transform A over F on the sources to a 
given receiver then the transform for Gaussian case is given by A which has the same entries as A but operations 
are over the reals. Since we assume A is full rank then A is full rank as well. Thus, we can solve for each original 
source at every receiver at distortion 

Di = a( ) (72) 



N x + P x 

a = 7 2 (\V N \ + IVmacIFupperK? - I) 2 - (73) 



22 



Finally, we invoke Lemma [5] to get a multicast rate from these distortions. We get that we can achieve any 
multicast rate satisfying: 

R<^o g ( a (^±^) I (74) 



2£ °\ V 

=I log ( 1+ f)-l loga (75) 

7 

= 7^ - ^ log a (76) 

= c '- 5 2-¥^ a (77) 

Choose £ large enough such that we can achieve R = C — 5. By making all the appropriate blocklengths large 
enough, we can make the probability of error arbitrarily small. This completes the proof. □ 



Acknowledgments 

The authors would like to thank G. Bresler, U. Erez, K. Narayanan, A. B. Wagner and M. P. Wilson for valuable 
discussions. The work of M. Gastpar was supported by the National Science Foundation under CAREER Grant 
CCF-0347298 and the work of B. Nazer was supported by a National Science Foundation Graduate Research 
Fellowship. 



References 

[1] R. Ahlswede, "Multi-way communication channels," in Proceedings of the 2nd International Symposium on Information Theory, Prague, 

pp. 23-52, Publishing House of the Hungarian Academy of Sciences, 1971. 
[2] H. Liao, Multiple access channels. PhD thesis, University of Hawaii, Honolulu, 1972. 

[3] T. M. Cover, "Comments on broadcast channels," IEEE Transactions on Information Theory, vol. 44, pp. 2524-2530, October 1998. 
[4] T. M. Cover and A. El Gamal, "Capacity theorems for the relay channel," IEEE Transactions on Information Theory, vol. 25, pp. 572- 
584, September 1979. 

[5] J. Korner and K. Marton, "How to encode the modulo-two sum of binary sources," IEEE Transactions on Information Theory, vol. 25, 
pp. 219-221, March 1979. 

[6] R. L. Dobrushin, "Asymptotic optimality of group and systematic codes for some channels," Theory of Probability and its Applications, 
vol. 8, no. 1, pp. 47-59, 1963. 

[7] R. Ahlswede, "Group codes do not achieve Shannon's channel capacity for general discrete channels," The Annals of Mathematical 

Statistics, vol. 42, pp. 224-240, February 1971. 
[8] B. Nazer and M. Gastpar, "Computation over multiple-access channels," IEEE Transactions on Information Theory, vol. 53, pp. 3498- 

3516, October 2007. 

[9] A. D. Sarwate, B. Nazer, and M. Gastpar, "Spatial filtering in sensor networks with computation codes," in Proceedings of the 2007 

IEEE Statistical Signal Processing Workshop, (Madison, WI), August 2007. 
[10] B. Nazer and M. Gastpar, "Structured random codes and sensor network coding theorems," in International Zurich Seminar on 

Communications (IZS 2008), (Zurich, Switzerland), March 2008. 
[11] D. Krithivasan and S. Pradhan, "Lattices for distributed source coding: Jointly Gaussian sources and reconstruction of a linear function," 

IEEE Transactions on Information Theory, Submitted July 2007. See http://arxiv.org/abs/0707.3461 
[12] T. Philosof, A. Khisti, U. Erez, and R. Zamir, "Lattice strategies for the dirty multiple access channel," in Proceedings of the IEEE 

International Symposium on Information Theory (ISIT 2007), (Nice, France), June 2007. 
[13] V. R. Cadambe and S. A. Jafar, "Interference alignment and the degrees of freedom for the K user interference channel," IEEE 

Transactions on Information Theory, Submitted July 2007. See http://arxiv.org/abs/0707.0323 
[14] G. Bresler, A. Parekh, and D. Tse, "The approximate capacity of a one-sided interference channel," in 45th Annual Allerton Conference, 

(Monticello, IL), September 2007. 

[15] J. Andrews, N. Jindal, M. Haenggi, R. Berry, S. Jafar, D. Guo, S. Shakkottai, R. Heath, M. Nealy, S. Weber, and A. Yener, 
"Rethinking information theory for mobile ad hoc networks," IEEE Communications Magazine, Submitted December 2007. See 
|http://arxiv.org/abs/0712.2467"1 

[16] A. Sanderovich, M. Peleg, and S. Shamai, "Scaling laws in decentralized processing of interfered Gaussian channels," in International 

Zurich Seminar on Communications (IZS 2008), (Zurich, Switzerland), March 2008. 
[17] B. Nazer and M. Gastpar, "Computing over multiple-access channels with connections to wireless network coding," in Proceedings of 

the IEEE International Symposium on Information Theory (ISIT 2006), (Seattle, WA), July 2006. 
[18] B. Nazer and M. Gastpar, "Lattice coding increases multicast rates for Gaussian multiple-access networks," in 45th Annual Allerton 

Conference, (Monticello, IL), September 2007. 
[19] K. Narayanan, M. P. Wilson, and A. Sprintson, "Joint physical layer coding and network coding for bi-directional relaying," in 45th 

Annual Allerton Conference, (Monticello, IL), September 2007. 



23 



[20] W. Nam, S.-Y. Chung, and Y. H. Lee, "Capacity bounds for two-way relay channels," in International Zurich Seminar on Communications 

(IZS 2008), (Zurich, Switzerland), March 2008. 
[21] M. Gastpar and M. Vetterli, "On the capacity of large Gaussian relay networks," IEEE Transactions on Information Theory, vol. 51, 

pp. 765-779, March 2005. 

[22] Y. Kochman, A. Khina, U. Erez, and R. Zamir, "Rematch and forward for parallel relay networks," in Submitted to 2008 IEEE 

International Symposium on Information Theory, 2008. 
[23] B. Nazer and M. Gastpar, "The case for structured random codes in network communication theorems," in Proceedings of the IEEE 

Information Theory Workshop (1TW 2007), (Lake Tahoe, CA), September 2007. 
[24] T. M. Cover, "A proof of the data compression theorem of Slepian and Wolf for ergodic sources," IEEE Transactions on Information 

Theory, vol. 21, pp. 226-228, March 1975. 
[25] T. M. Cover and C. S. K. Leung, "An achievable rate region for the multiple-access channel with feedback," IEEE Transactions on 

Information Theory, vol. 27, pp. 292-298, May 1981. 
[26] T. M. Cover, "Broadcast channels," IEEE Transactions on Information Theory, vol. 18, pp. 2-14, January 1972. 
[27] D. Slepian and J. Wolf, "Noiseless coding of correlated information sources," IEEE Transactions on Information Theory, vol. 19, 

pp. 471-480, July 1973. 

[28] R. Zamir, S. Shamai, and U. Erez, "Nested linear/lattice codes for structured multiterminal binning," IEEE Transactions on Information 

Theory, vol. 48, pp. 1250-1276, June 2002. 
[29] S.-Y. R. Li, R. W. Yeung, and N. Cai, "Linear network coding," IEEE Transactions on Information Theory, vol. 49, pp. 371-381, 

February 2003. 

[30] R. Koetter and M. Medard, "An algebraic approach to network coding," IEEE/ACM Transactions on Networking, vol. 11, pp. 782-795, 
October 2003. 

[31] R. Dougherty, C. Freiling, and K. Zeger, "Insufficiency of linear coding in network information flow," IEEE Transactions on Information 

Theory, vol. 51, pp. 2745-2759, August 2005. 
[32] P. Elias, "Coding for noisy channels," IRE Convention Record, vol. 4, pp. 37-46, 1955. 
[33] R. Gallager, Information Theory and Reliable Communication. New York: John Wiley and Sons, Inc., 1968. 

[34] I. Csiszar and J. Korner, Information Theory: Coding Theorems for Discrete Memoryless Systems. New York: Academic Press, 1982. 
[35] I. Csiszar, "Linear codes for sources and source networks: Error exponents, universal coding," IEEE Transactions on Information 

Theory, vol. 28, pp. 585-592, July 1982. 
[36] R. Urbanke and B. Rimoldi, "Lattice codes can achieve capacity on the AWGN channel," IEEE Transactions on Information Theory, 

vol. 44, pp. 273-278, January 1998. 
[37] U. Erez and R. Zamir, "Achieving | log (1 + SNR) on the AWGN channel with lattice encoding and decoding," IEEE Transactions 

on Information Theory, vol. 50, pp. 2293-2314, October 2004. 
[38] R. Zamir and M. Feder, "On lattice quantization noise," IEEE Transactions on Information Theory, vol. 42, pp. 1152-1159, July 1996. 
[39] U. Erez, S. Litsyn, and R. Zamir, "Lattices which are good for (almost) everything," IEEE Transactions on Information Theory, vol. 51, 

pp. 3401-3416, October 2005. 

[40] H.-A. Loeliger, "Averaging bounds for lattices and linear codes," IEEE Transactions on Information Theory, vol. 43, pp. 1767-1773, 
November 1997. 

[41] Y. Kochman and R. Zamir, "Joint Wyner-Ziv/dirty-paper coding by analog modulo-lattice modulation," IEEE Transactions on 

Information Theory, Submitted January 2008. See http://arxiv.org/abs/0801.0815 
[42] T. Cover and J. Thomas, Elements of Information Theory. New York: Wiley-Interscience, 1991. 

[43] B. Nazer and M. Gastpar, "Computation over Gaussian multiple-access channels," in Proceedings of the IEEE International Symposium 

on Information Theory (ISIT 2007), (Nice, France), June 2007. 
[44] R. Ahlswede, N. Cai, S.-Y. R. Li, and R. W. Yeung, "Network information flow," IEEE Transactions on Information Theory, vol. 46, 

pp. 1204-1216, July 2000. 

[45] L. R. Ford and D. R. Fulkerson, Flows in Networks. Princeton, NJ: Princeton University Press, 1962. 

[46] T. Ho, D. R. Karger, M. Medard, and R. Koetter, "Network coding from a network flow perspective," in Proceedings of the IEEE 

International Symposium on Information Theory (ISIT 2003), (Yokohama, Japan), June 2003. 
[47] S. Jaggi, P. Chou, and K. Jain, "Low complexity algebraic network codes," in Proceedings of the IEEE International Symposium on 

Information Theory (ISIT 2003), (Yokohama, Japan), June 2003. 
[48] P. Sanders, S. Egner, and L. Tolhuizen, "Polynomial time algorithms for network information flow," in 15th ACM Symposium on Parallel 

Algorithms and Architectures, pp. 286-294, 2003. 
[49] T. Ho, M. Medard, R. Koetter, D. R. Karger, M. Effros, J. Shi, and B. Leong, "A random linear network coding approach to multicast," 

IEEE Transactions on Information Theory, vol. 52, pp. 4413-4430, October 2006. 
[50] R. G. Gallager, "A simple derivation of the coding theorem and some applications," IEEE Transactions on Information Theory, vol. 11, 

pp. 3-18, January 1965. 



